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Browsing,  Discovery  and  Search  in  Large  Distributed  Databases  of  Complex  and 

Scanned  Documents 

Technical  and  Scientific  Report 


Task  1:  Representation  Techniques  for  Complex  Documents 

Task  Objectives 

In  this  task,  the  goal  is  to  extend  the  word-based  representations  that  are  common  in 
retrieval  systems  in  order  to  support  summarization,  browsing,  and  more  effective 
retrieval.  Specifically,  we  will  be  studying  phrase-based  representations  and  relationships 
between  phrases  in  individual  and  groups  of  documents  as  the  basis  for  our  approach. 
Document  structure  will  be  used  as  part  of  the  information  that  is  used  to  "tag"  the 
phrasal  representation. 

Technical  Problems 

The  technical  problems  have  to  do  with  defining  a  "phrase",  developing  techniques  for 
rapidly  extracting  them  from  text,  comparing  phrase  contexts  to  identify  significant 
relationships,  producing  summaries  from  these  representations,  extending  the  underlying 
retrieval  model  to  be  able  to  make  effective  use  of  phrasal  representations,  and  using 
complex  document  structure  in  indexing  and  retrieval. 

General  Methodology 

The  general  methodology  for  this  task  is  to  demonstrate  effectiveness  through  user-based 
and  collection-based  experiments.  Extensive  use  will  be  made  of  the  TIPSTER  document 
collection,  which  consists  of  a  large  number  of  text  documents  from  a  variety  of  sources, 
queries,  and  user  relevance  judgments  for  each  query.  We  will  also  be  making  increased 
use  of  PTO  text  databases  in  these  experiments. 

Technical  Results 


The  new  phrase  indexing  approach  was  applied  to  patent  data.  This  demonstrated  that 
patents  make  heavy  use  of  phrases  and  that  the  phrases  are  substantially  different  than 
those  found  in  the  TREC  database  (examples  in  overheads  attached).  A  new  patent 
retrieval  demonstration  incorporating  this  indexing  was  shown  at  the  second 
DARPA/PTO  status  review  meeting  in  Washington  D.C.  on  the  10*^  December.  In 
addition,  for  this  demonstration  another  year  of  patents  was  indexed. 
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Important  Findings  and  Conclusions 

Initial  results  show  that  phrase  indexing  and  query  formulation  techniques  substantially 
improve  the  results  of  patent  searches. 

Significant  Hardware  Development 

None 

Special  Comments 
None. 

Implication  for  Further  Research 

We  plan  to  continue  to  enhance  the  query  processing  and  retrieval  strategies  for  patents, 
including  the  use  of  automatic  query  expansion  techniques.  We  also  plan  a  version  of  the 
patent  search  demonstration  with  an  improved  user  interface  to  integrate  Boolean  and  free 
text  searching. 


Task  2;  Browsing  and  Classification  Techniques  for  Document  Collections 
Task  Objectives 

The  goals  of  this  task  are  to  develop  techniques  for  summarizing  and  classifying 
collections  of  documents.  These  techniques  will  be  designed  to  support  interactive 
browsing  and  text  classification  in  environments  like  the  PTO. 

Technical  Problems 

The  technical  problems  involve  producing  an  effective  summary  of  a  group  of 
documents,  such  as  a  retrieved  set  or  an  entire  database.  Both  document  and  phrase 
clusters  could  be  used  as  part  of  this  process.  The  classification  task  emphasizes  the 
ability  to  accurately  assign  predefined  categories  (as  in  the  PTO  classification)  to  new 
documents  (patents).  An  additional  problem  is  to  determine  when  existing  classifications 
do  not  match  well  to  new  documents,  such  as  when  a  PTO  category  covers  too  many 
patents  and  needs  to  be  refined. 

General  Methodology 

Evaluation  of  these  techniques  will  be  done  using  both  the  TREC  corpus  and  PTO  data. 
For  the  classification  task  in  particular,  we  are  designing  evaluation  criteria  with 
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substantial  input  from  PTO  staff.  Most  of  the  classification  experiments  will  be  done  in 
the  context  of  the  PTO  classification  and  previously  classified  patents. 


Technical  Results 

The  TREC  evaluation  of  an  approach  to  visualizing  retrieval  results  showed  that  some 
users  were  able  to  obtain  significant  retrieval  benefits.  A  discussion  took  place  at  the 
December  DARPA/PTO  review  about  which  visualizations  may  be  the  most  useful  for 
patent  searching. 

An  on-line  demonstration  of  the  patent  classification  system  was  given  at  the  December 
review  meeting.  This  demonstration  showed  that  nearest  neighbor  classification  based  on 
full  patent  searching  can  produce  very  good  results. 

Important  Findings  and  Conclusions 

The  TREC  evaluation  was  one  of  the  first  of  this  scale  for  this  type  of  information 
visualization.  Our  results  continue  to  indicate  that  many  classes  of  patents  could  be 
reliably  classified  automatically. 


Significant  Hardware  Development 
None 

Special  Comments 
None. 

Implication  for  Further  Research 

We  are  now  focusing  on  evaluating  the  classification  accuracy  and  incorporating 
additional  classification  techniques  into  the  classification  system. 

Task  3:  Image  Indexing  and  Retrieval 

Task  Objectives 

The  goal  of  this  task  is  to  develop  similarity-based  techniques  for  retrieving  images  such 
as  trademarks,  logos,  and  designs. 

Technical  Problems 

The  central  issue  is  how  images  can  be  indexed  to  support  efficient,  content-based 
retrieval.  The  primary  type  of  query  in  these  environments  is  “find  me  things  that  look 


-3- 


like  this”.  We  are  developing  "appearance-based"  retrieval  of  images  as  well  as  more 
straightforward  features  such  as  color  and  texture.  Filter  based  and  frequency  domain 
based  techniques  offer  some  potential  in  this  area,  but  significant  work  needs  to  be  done 
on  making  this  approach  efficient  enough  to  deal  with  hundreds  of  thousands  of  images. 

General  Methodology 

The  evaluation  of  these  techniques  will  be  done  in  a  similar  way  to  text  by  developing 
test  collections  of  images.  Specifically,  we  are  working  to  obtain  large  collections  of 
trademark  and  design  images,  both  from  the  PTO  and  from  general  sources  such  as  the 
web. 

Technical  Results 

The  new  image  retrieval  techniques  were  applied  to  a  much  larger  (60,000  image) 
trademark  database  and  demonstrated  in  a  multimodal  text  plus  image  trademark  retrieval 
system  at  the  December  meeting.  Some  problems  were  found  in  that  initial  demonstration 
have  since  been  fixed.  Initial  studies  of  flower  patent  retrieval  were  also  presented  at  the 
meeting. 

Important  Findings  and  Conclusions 

The  feedback  from  the  December  meeting  clearly  indicated  the  benefits  of  combining  text 
plus  image  retrieval.  Important  directions  for  improving  the  system  were  also  discussed. 

Significant  Hardware  Development 

None 

Special  Comments 
None. 

Implication  for  Further  Research 

We  will  continue  to  improve  the  demonstration  trademark  retrieval  system  by  refining  the 
text  search  component  and  refining  the  image  match  algorithms.  We  also  plan  to  further 
scale  up  the  system  to  handle  hundreds  of  thousands  of  trademarks. 

Task  4:  Distributed  Retrieval  Architecture 

Task  Objectives 

The  goals  of  this  task  are  to  scale  up  our  current  methods  of  automatically  selecting 
collections  and  merging  results,  and  to  investigate  architectures  that  can  support  efficient 
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retrieval,  browsing  and  relevance  feedback  in  distributed  environments  with  terabytes  of 
information. 

Technical  Problems 

The  current  INQUERY  text  retrieval  system  uses  a  client  server  architecture  to  support 
simultaneous  retrieval  from  multiple  collections  distributed  across  one  or  more 
processors.  A  number  of  efficiency  bottlenecks  develop,  however,  when  the  size  of  the 
databases  is  very  large.  Deciding  which  subcollections  to  search  can  address  part  of  the 
problem,  but  there  are  other  problems  associated  with  the  fundamental  efficiency  of  the 
processes  involved  and  the  use  of  distributed  resources.  Image  indexing  and  retrieval 
tends  to  make  all  of  these  problems  worse  since  the  databases  and  indexes  are 
considerably  larger. 

General  Methodology 

The  architectures  and  algorithms  produced  in  this  task  will  be  evaluated  using  a 
combination  of  standard  performance  (efficiency)  measures  and  effectiveness  measures. 
The  efficiency  tests  will  be  done  using  TREC  data  and  large  PTO  databases,  including 
images,  and  the  collection  selection  algorithms  will  be  evaluated  using  the  text 
subcollections  of  the  patents. 

Technical  Results 

More  experiments  on  distributed  search  were  carried  out.  The  DS3  connection  to  AAINet 
was  finally  installed  in  December. 

Important  Findings  and  Conclusions 


None.  - 


Significant  Hardware  Development 


Special  Comments 
None. 

Implications  for  Further  Research 

We  are  currently  working  on  defining  a  distributed  search  experiment  that  would  involve 
having  one  or  more  servers  at  the  San  Diego  site. 


-5- 


^4r/  JO^ 


(/) 


GC 

O 


Center  for  Intelligent  Information  Retrieval 
University  of  Massachusetts,  Amherst 

[MIC  QUALITY  mSPECTED  8 


Overview  of  Tasks 


I 


X  cn 

(D  (D 


r  ^ 

?H  QJ 


a  ^ 

■■m  s 

iS  « 
s  5P 
^  S  2 

Vi  ’S 
cu  ^  o 

P-i  o  ;Ih 

CQ  u 


03 

C  > 

03  5^ 

•  t-H 

U  0) 

X 

O)  ’Id 


a;  713:^ 
60  H 


86/S/I. 


Search  Scenarios  (Text  and  Image) 


40 


I 


CO 


(D 

C/D 

03 

rO 

cc3 

o3 

4-» 

a 

O) 


03  X 
Ph  W 


C/D 

<D 

C/D 

03 

4-» 

03 

"d 

d 

O 


O) 

C/D 

03 

4-> 

03 

4-> 

d 

O) 

4-> 

03  X 

Ph  W 

A  A 


C/D 

O) 

C/D 

o3 

ri::) 

03 

03 

d 

O) 


A 

A 

1 

> 

1 

1 

u 

0) 

u 

CD 

U 

<D 

P 

CD 

P 

d 

d 

CT^ 

CP 

cr 

CT^ 

4-> 

d 

4-» 

P 

CD 

u 

CD 

CD 

CD 

4-> 

P 

P 

cc3 

cc3 

CD 

CD 

Ph 

P^ 

o 

o 

o 

• 

• 

• 

CIIR  1/5/98 


Representation  Techniques 


I 


o  s 

^  6JD 

S  ^ 

a  I 

o  o 

xJ  ^ 


!=;  n 

O)  O 

C/D  *1-^ 
;l_|  Cu 

<D  ?H 

t3  g 


-9  S 

-  S 

I  ^ 

>  Oh 

a 

X  S' 

w  ^ 

S 

O  H-I 


o  ^ 


52  'iJ 

2 

^  § 

Oh  g 


id  (L» 

u  C/D  *1^ 
0  0)4^ 

-M  TO  ;H 

S  4:;  bJO 

■a  U.g 

- 

o  C  ;h 

dj  *t>.  ^ 

I  "S  3 

«  2  !=i 

*1—1 

•  |-ai(  •  «  r'  f  ^ 

o  (x»  y 
7^  TO  fn 

^  I  - 

W  ^  X 

. .  ^  a; 

''TO  I  ■  < 

cu  frt  •s 

O  Tj 

wrv  W  J_J 


d  ^ 
.3  > 

C/D  (D 

^  Q 

u  ui 
O  d 

?H  U 

PL^  O 


^  "s 

d  1 
cr*  ‘ 


86/S/l  yil3 


Representation  Techniques 


m 


cd 

•S 


•  rH 
rO 

o 


^  32 

U  Co 

60 

o  ctS 

Ph  ^ 


o 

>  rH 

4-> 

>  rH 

cn 


QJ 

60 

;h 

CTS 


^  o 

U  O 

03 


03 

u 


X 

O) 


03 

a 

O 


CJ 


Ti 

(D 

O  O) 

(/) 


CD 


03 


P  a, 


'4h 

o 

I 

-M 

03 

Cl, 

c 

03 

rC^ 

a; 

-♦-> 

o3 

v-i 

u 

u 

03 

<D 

Jh 

O 


t3 

p; 

o3 

Jh 

CD 

4-» 

cn 

o3 


rli: 

u 
o3 
O 

;p 

^  *B^ 

bO 


cn 

C^ 

O 

•  rH 
4H 

03 

u 

'o 

u 

bJO 

.a 

•4-> 

cn 

<D 

;h 

<D 


O) 

nd 

P! 

u 

X 

O) 


Cl. 

o3 


t3 
a> 
bO  t3 
<D 
CD 

a 


bO 

o3 


03 

.Jh  ^ 
1b  ^ 


cn 

CJ 

•  rH 
+-» 

cn 

•  iH 

Pi 


CD  CD 


(D 

d 

03 

•  rH 

c/3 

CD 

03 

CQ 

?H 

O 

Mh 

c/3 

Jh 

o 

4-» 

03 

Pi 

(D 

O 

Mh 

O 

c/3 

C/3 

03 


•D 

CD 

a. 

O 


^  o;  CD 

r-M  ^  p 


CD 


>  ^ 
S  o 

Q  S 


03 

a 

o3 

-P 

o 

u 

C3 

<D 

03 

•  rH 

0) 


o 

a 


o3 

C3 


O 

cj 


CD 
bO 
o3 
p 
CD  CD 


bJO 

C3 


O) 

p 

CD 


> 

03 

03 

.i=i 

CD  rP3 

jp  5X3 

^  t 

JD  cb 
o3  2^ 

S 


cc 

o 


shown  to  be  useful  in  modeling  Boolean  combinations 


Phrase  Extraction 


(D 


^  g 

^  u 

H  N 
^  H 

^  pCn 

B 

o  S 


X  ^ 

CL)  O 
C/D  8 
^  CD 

5S  O 

2  O 

Oi  e 


o  X 
o 


2  ^ 
n 

S  ^ 


u  l>^  d 

^  5r1  n 

4-'  ^ 

^  2 

-  -S  I 

rt  O 

0->  r- 

b  C/D  6 


Si  > 
o  g  .a> 

O  in  S_| 

V  r  "i  Q-' 

CO  ^  ?H 


86/S/l.  yi'3 


Top  Phrases  from  TIPSTER 


C/3 

<V 

CJ 

O 


c 

O 

52  gffl 
c  N  ^ 


gs 

<  s. 


u 

cu 


C 


c: 

B 

B 

o 

<J 

c 

o 

•IM 

a 


^  a 

^  C  fl  C 

c  3  c3  _  P 

«  o  Oi  J3  § 

^  rj  u  u  jP. 

•s  ^  O  «  U 

r  •4-»  «3  m 


OX) 

0 

E 

p 

p 

UAJ 

•  PN 

0 

'S 

Vi 

W3 

u 

p 

u 

0 

Cu 

0 

u 

p 

X! 

u 

OX) 

c 

u 

p 

p 

U4 

p 

0 

0 

0 

ZJ 

a 

0 

0 

<u 

2 

< 

CO 

p 

P3 

0 

CO 

Z  P 

U 

00 

VO 

VO 

»v 

00 

cs 

CO 

00 

CO 

rH 

CO 

rH 

CS 

10 

lo 

tT 

rt 

Os 

cs 

00 

i> 

1-* 

VO 

VO 

to 

Tf 

rr 

cs 

0 

0 

Os 

10 

lO 

10 

lO 

10 

»o 

m 

10 

»o 

10 

in 

"rr 

c 

<1^ 

E 

O 

-S  « 

O  '^2 

*s  P 

C  O) 

no 


o 
c 
o 

^  cu 

fS  00 

r-  o 

QO  QO 
Tf  Tf 


c 

<U 

5  «*-i 

2  "S 

^  -a 

C/}  ^ 

U  ^ 

a<  C 

ri  fS 
IT)  o 

\0  VO 

Tf  Tf 


»3 

•iP4 

ZJ 

P 

P 

c 

P 

P 

0 

0 

0 

0. 

u 

p 

u 

P 

p 

r/j 

o 


^  5? 

O  ^  fc- 
^  CJ  ^ 

g  ^  cw  5 

i  aZ  § 

w  C  P  (73 

»rt  VJ  00  VO 
'O  Ov  ■<»  <<» 
l«  ■ct  rt 
TT  Tf 


A 

fe  "^  § 

A  01)  d 

S.  5  Q 

5^  « 

to  w  *5 

un 

0\  ^  t> 
rH  VO  O 

<<1  t-H  rH 
Tf  Tf  Tf 


>> 

E  .H 
a  o  B 

a  C  . 
c  o  S 

O  «  ^ 

U  ^  lo 
t/j  E  !> 

I/)  Cv  rH 
00  VO  Tf 
O  0\  ON 
rf  r)  CO 


00 

LO 


C« 

o 


■3  a  2  M  « 

-2  >>  a  a  o 

^  2P'5 
■a  —  5  *  j- 

«  o  bi)£ 

'S  t:  o  o  o 

p  ^  p  a  z 

TT  <S  QO 
cs  VO  VO  QO 

00  CO  QO  O 

lo  i-H  CO  00  r^ 

VO  VO  CO  rH  tH 


(U 

E 


E  - 

P  .22 

^  o  CJ  •£ 
is  W)^  c 

O  (U  4>  C 

>H  ur  0J3 

^  fl  c« 

4J  ^  A  ^ 
Z  CO  O  a  c: 

00  CO  ^  Os  ON 

O  rH  O  VO  ON 

cotnooor- 
t-*  lo  lo  cs  cs 


c 

c 


> 

o 

CO 

l> 

VO 

o 

cs 


fi 

•2 

c« 

u 

0) 

-'O 

c 

.2 

‘So 

W3 

5 


.2 

'S 

u 

Vi  c2 

c  ;z: 
o  C«  C3 
'•5  U  2 

^  C  O 

^  JZ  JZ 
^  ^  ^ 
•300 
s  o  o 
P  CO  CO 


p 

o 


OX)  •: 


^  2 
^  c  2 

S  2  ^ 

0^  Cli  f-t 

0’S 

C2  P  O 

§  W 


o 

zj 

Vi 

'3 

p 

p 

p2^ 


o 

o 

C 

CJ 

u 

•S 

o 

o 

C/3 

0) 

p 


CA 


c/3 

p 

o 


Vi 
Vi 

2  w) 
2  u 
04 


rH  cs  r-  O  O  Tf  VO 

00  rH  CS  Tf  CS  CS  CO 

O  ON  rH  VO  VO  10  Tf 

rH  0\  00  t--  t^ 


p 
p 
CO 
CS  VO 
VO  QO 
CO  O 


Si 

cs  00 

ON  TT 


r-' 

VO 


CO 

VO 


o 

Vi 

P 
o 

a 

o 

2 

t--  »o  l> 
10  10  CO 
rH  00 
VO  »0  10 


P 
P 

E 

eu  a 


O 

U 

P 

o 


QC 

O 


I 


Top  Phrases  from  Patents 


■  ^  tJ 

^ 

D<  CI4  CO 
UT  fS  UT 

m  ri 

10  <s  o 

^  ON  On 
<N| 


Q 

O 

n 

l-’o 

«  c  Q 
*5  e  S 

•s  &“ 

'O  E  M 
5  S  fa 
w)  ^  W 
S  ~  fftJ 
.2  -iS  fa 

^-H  cn  Tf 

VC  o  m 

ON  r-*  Tf 

00  r-  t-; 

ts  fS 


.2  S 

5>  fc-  « 

«  2  3 

-  c  fa 

CO  .Sn  M 

^  ro  <s 

00  o  o 

t-H  ON  00 
I/)  U1 
fS  r<l 


'T3 

O 

^  "2 

g  ai  E 

<u  <u 

«  C 
^  JS  '3 
S  Cm’S 

t>  m 

Tf  Tf  rH 

o  o 

IT)  TT 

fS  «S 


C3 

C 

W)  ^ 
*5 

^  o 

2 

i  o 

& 

®  & 

cj  O 


O 

-  g 

o  s  . 

M  CO 

•g  g  , 

S  a 

a>  'O 
fa  S 

aj  "D 

E  g 


< 

S 

u  -s 
W  2 
U  g 
Z  2 

M  fa 

P  s 

O'  2f 

a  'sj 

t/i  ^ 


o  g 

fa  o  ' 

^  CO 
O  ^ 

S  w) 

O  M  . 


a  Oi  c: 
r>  00  Tt 
o  rs  <s 
•nt  00  r- 

VO  ON  00 
VO  IT)  m 


1  ii  .2  § 

ft  'tJ  ^ 

-•  S  ^ 

S  cj  o 
^  S  o  o 
^  tCn  'S 

<U  w  jO  CO 

10  m  On 

tH  0\  O  ON 
rH  tH  O  <S 
Tf  «s  rl  VO 
1/7  10  UT  Tf 


5^  c 

C  tsD 
ft  “  'E 

^  'M  C 

S  3  « 

O  ft 

s  3  ,i5 

c«  O 

Tt  rr  rH 

ON  IT)  T-l 
VO  On 

r-(  o  r- 
Tf  tt  ro 


T3  «  .2 

a  'O  'O 
«  «  3  5 

'2  c  -  “ 

o  I2  “ 

g  ft. 22  2 

CO  C3  'T5  O 

T-(  00 

<s  00  Tf  m 
QO  00  Os  fO 
10  TT  m  <s 
r)  CO  ro  cn 


fi  g  ^ 

2  «  «2 

33  «{-  i2 

^  ^  ^ 
O  ^  CO 
CO  CO  / 

CO  J-j  g 

g  ij  ft 

^  2  & 

U  O  OS 

00  CO  UT 
m  Os  f<) 
CO  r-l  VO 
CS  O  ON 
CO  CO  cs 


Phrases  from  TREC  Queries 


(D 


<0 

JC  Ut 


a 

o 


y  o 

P  M  ^  2 


CL  , 

§ 

loo. 


0^0^ 


•5  ^ 

(U  c 


(13  c:  c  (D  ^  > 

^  ^  S  5 

;3  cd  g  o 

ij  X>  CL  i3  O  OJO 


in 


o 

cs  m  o  VO  r- 


cnr>cncnooovom^cooo 

^tniocnin^cNvo'^cnm 


VO  VO  OV 

CO  ooooinvomxfooc^  '^uo 
'^'^OOOVOCNOO^OCNVOOCO 
r-HOV’— '00’-HG^CNVnT— tT-HOOCS(N^^ 


o> 


86/S/I.  yil3 


Representation  Techniques 


O 


t 


O 

•  fH 

C/!) 

QJ 

QJ 

CD  U 

(jh  pq 

a  s 

H 


QJ 

u 

<u 

u 


I 

4-> 

X 

O) 

o 

u 

'"rt 

a 

a 

•T— I 


cn 

CD 


Oj 

p: 

» 

"cn 

Oh 

O) 

CD 

0 

o 

CD 

QJ 

U 

o 

u 


6J0 

0 

►  rH 


0 

CD 

"d 


o 

0 


c3 

0 

> 

CD 

13 


<D 

"d 

O 


CD 

c 


CD 


CD 


O 

D 

bO 

C 

•  rH 

D 

u 

S 

- 

H 

d 

«  rH 

"d 

D 

-4-> 

cn 

D 

H-* 

Sr^  cn 
Jh 

CD  ^ 

o-  ' 


0 

0 

d 

-4-» 

03 

rM 
O 

o 

S  W 

u  a  i 
o  z  § 

^  a 

tj  .5  - 

O)  W  0 

TS  3  0 

^  (1)  i 

^  3  i 

^  ^  I 

Q  xJ 


?H 

OJ 

u 

D 

Uh 

D 

U 

d 

03 

D 

I  I 

D 

Jh 

V 

C/^ 


DC 

O 


processing,  various  display  enhancements 


Clusters  from  Breast  Cancer  query 


& 


u 

lU 

C 

5  s 

S  u 


a. 

s 

o 

u 

O 


V 

OA 

3 

3 

3S  ^ 
0*^ 

3 

CO 

*-1 

O  « 
2  2 
*3  3 
3  3 
o  u 

0) 

3 

3 

1 

u 

4> 

U 

3 

3 

<U 

U) 

u 

3 

CO 

u  3 
3  *C 

3  S 
U  ;w 

3  2 

O  ^ 

0 

o 

'W 

u 

2 

£  3 

.2  2 
M  JO 

o 

3 

o 

£ 

JS 

X 

a> 

CO 

CO 

3 

a 

u 

..  P0 

4) 

O 

3 

3 

U 

.2 

3 

..  3 

3 

3 

.2 

CO 

JS 

CJ 

Lm 

el 

<u 

CO 

£ 

Ui 

4> 

O 

C 

a 

CJ 


00 

un 


a 

s 

o 

u 

O 


\o 

04 

s 

o 

u 

O 


a 

3 

o 

u 

O 


u 

*0 


c 

<u 


04 

3 

o 

u 

O 


fS 

01 

3 

O 

u 

a 


2  1“ -S  I 

-S  5  fl  o 
ti  ^  o 
O  0)  X  V 

S  S-  g-'K 

^  £  £  A 

o  ^  2;*  £ 


>> 

0 

e3 

u 

4J 

£ 

3 

.2 

2 

3 

U 


<ii  a 

2  ^ 
M 

fc  .i 
3  t 
2  3 


3 

iS 
E  . 


U) 


0 

3 

O- 

u 

o 


3 

2 

.0 


^  O 

On  .2 

.§  ^ 


5 

o 


silicone  gel  breast  implant 
silicone  implant 


TREC  Query  Clusters 


ca 


00 

05 

LO 


5 

O 


listeria 


Determining  Core  Concepts 


ja 


cn 


P-I 

1— H 

cd 

o 

•  tH 

o3 

12 


I 


cc 

o 


Use  combination  of  linguistic  analysis,  weighting, 
and  corpus  analysis  of  query  word  relationships 


Browsing  and  Classification 


s  ^ 

p,  > 


r2  ^ 


bp  ^ 

•rH  C 

.n 

M—l  1  I  ■ 


C/D  f-* 

(U  u 

3  oj 

V4— I 

cr  ^ 

rH  O) 


CCS 

,y  o 


tn  H 

C/D 


23  ^ 

C3  > 


^  M 

P  (D 


60  03 

^  w  r* 


.3  60 

^  n 


§  o 

o  .a 


*  o 

CO  i; 


pH  ^ 


3 

g 

CO  '"rt  cj 

xi  2 

a 

c«  w  g 

bo  9-1  ‘Xj 
C  g  1c 
•s  g  .9 


N  CCS 


CJ 

60 

C 

•  rH 

C/D 

P 


S  o 


2 

^  &• 


o  15 

pH 

o  ^ 


*-i  § 

P  pq 

- 

Q)  C/D 


C3  O 
N  u 


-4->  Ij 

OS  U 


3  ^ 

a  -S 

^  CO 

>  p 


o  o 
PH  rO 


^  rS 

»rH  CO 


testbed 


u 

O 


O 

O 

4-> 

C 

o 

•  ^ 

-M 

cd 

N 

•fH 

«/) 

•rH 

> 

C/D 

CJ 

•  ^ 

c^  n 

T  .  •rH 

So  ^ 
P  o 

CO  ^ 

a; 

(D  > 

pH 

o  u 

(D  S 

>  S 

<D 

P 


u 

u 

a 

0) 

CD 

bJD 

a 

•  r— I 

^  > 

cz  n 

o  ■ 
cn  d 


U 

w 

H 

d 

•  rH 

CD 

d 

o 

d 

o 

•  1— < 

4-> 

a 

13 

> 

CD 


u 


CD 

-4-» 

a 

u 

-+-» 

C/D 

d 

o 


<i> 

"d 


CD 


p 

•M 


E  C 
d  I  I 


d 

o 

d 

-M 

;h 

o 

M-( 

-M 

f3 

u 

•  rH 
'HH 

•  i^H 

"P-: 

C/D 

C/D 

r3 

7j 

p 

o 

CO 

cu 

d 

o 

CD 

o 

H 

CD 

0) 

4-> 

M-( 

pH 

o 

(D 

CJ 

d 

nd 

d 

o 

0) 

d 

d 

CCS 

Ti 

CD 

"P 

•  irH 

C/^ 

;u( 

OJ 

s 

'4— » 

c 

(d 

O) 

p 

rH 

OJ 

> 

CD 

'4H 

O) 

;h 

d 

"d 

►> 

cn 

5h 

•  f— ^ 

rd 

CD 

CCS 

Ph 

U 

> 

PH 

H 

1 

p 

1 

1 

<o 


00 


5 

O 


Demonstration  system  built 


Aspect  InQuery 


86/s/i.  yiio 


Assisted  Cluster  Browsing 


Browsing  and  Classification 


c/D 

O) 

c/D 

oS 

o  -2 

^  rS 
U  ^ 

03 
cn 
cn 
CD 


03 

I  I 

s 

03 

;-i 

6J0 

O 

Ph 

p 

o 


cn 

o 

'4H 
•  rH 

cn 

cn 

g3 


t3 

cd 


<D 


o3 


N 

•rH 

O) 

H-» 

o 

C/!) 

60 

OJ 

H-> 

CD 

4-> 

r— H 

03 

CO 

u 

O 

•  1— I 

CD 

> 

CD 

cS 

M-l 

Jh 

Ph 

(D 

1 

u 

03 

Sh 


03  bJD 

^  .s 


O 

;h 

u 

w 

H 

•4-» 

P 

O) 

u 

CL) 


t3 

CL) 

+-• 

CO 

O) 


u 

O 

•  rH 

cj  O 

u  ^ 

bJD 

•  rH 

«  c 

03  ^ 


CO 

<D 

03 

PQ 

o 

:§ 

bJO 

•  rH 

CD 

+-* 

CO 

CD 

u 

cc3 


CO 

O) 

;c-( 

cc3 

CD 

C 

{=: 

O 

CO 

•  rH 

CO 

u 

O 

M-l 

"iS 

•  rH 


I  I  I 


CIIR  1/5/98 


Image  Indexing  and  Retrieval 


O 

CNJ 


O  ^ 
cfi 

cn  o 

S-s 

•Jlj  V 

6 

<D  >735 

rr\  ^ 


c§  ^ 

K.  cn 

^  ci 

•  rH  ^ 

os  u 

•  rH  1— ^ 

6  S 

S  o) 


pL, 

O) 

Q 


bJO 

03 


6J0 

P 

►  t-H 

•  •  OJ 

s 

o  -<-• 

O) 


u 


C3 

4-* 

03 

t3 

13 

p 

CD 

-M 

X 

O) 

T3 

P 

03 

o 

H 

Ph 

-t-j 

o 

<D 

C/D 

•  • 

I  "  ■  '"^ 

o3 

g  §D 

CD  ^■' 
d)  I 


I  ^ 

-9 

rn  (D 

CJ  2 
P  rp 

c3  ^ 

Jrt  ^ 

Cu  ^ 

^  n 

03  O 


-  O 

dj  u 

•  1— {  OJ 

'ef 

U  M 

O  ^ 
rr ,  CD 
^  0) 

o3  t-h 

•1—1  03  s. 


O) 

•t-H 


~0  Pm  jj 

Ph  % 

c;3  ?H 


P 

o3 

u 

■4-J 

03 

cn 

cu 

•  rH 


CJ 

a> 


I  t 

o3 
t3 

i 

s  y 

'JU  cd 
"P  OJ 

§  ^ 
P  ^3 

Ph  S 

O 

’TT)  ^ 

>  ^ 
^  c 


Q 


d5  ^ 

8  p 

o 

'n  ^ 

5 

CO  O) 


UJ 

Lf5 


CC 

O 


very  large  databases  of  images 


Image  Indexing  and  Retrieval 


c3 

QJ 


Q) 

U 

QJ 

bJD 


"d 

d) 

CD 

03 

CD 

CJ 

d 

CCS 

?H 

03 

O) 

Ph 

Ph 

CCS 


(D 

P 

CD 

0  a; 

Sh 

CD  U 

Q 


bO 


.a 

cb 


(D 

60 

ct3 

g 

•  t-H 

lb  ^ 
■4J  60 

b  c 

CCS 

Q.  i-P 
^  U 


?H 

o 


c6 


O) 

60 

cd 


cn 

.2 
«  R 
Qh  (U 

P  "o 

o  o 


CD 


"d 

P 

CCS 

U 

CCS  CCS 

TS  S 

o)  1:3 

60  ^ 

CCS 


CCS 

j-i 


42  ^ 

CD  CD 

CL)  CCS 


p 

CD 

> 


LO  0)  4S 

I  I 


CCS 

?H 
4-> 

o 

Qj  Cl, 

O)  ll 

a 

O  oS 

TS  O 

<D  li 

CD  P 


O 

Q 


d 

CCS 

d 

P 

CCS 

4-» 

CD 


CD 

CD 

60 

CCS 


Jh 

CCS 


CD 

d 

CCS 

L 

H 

d 

p 

CCS 


CD  ^ 
CD  O 
bJD  O 


CT!) 

p 

o 

CD 

P 

_P 

”0; 

CJ 

CD 


O) 

d 

CD 

-4-' 

?-( 

(D 

P  ILP 

o  G 

CJ  H 


d 

CD 

cS 

CD 

CD 

-M 

5h 

O 

mh 

CD 

CCS 


CD 

d 

CCS 

?-i 

-4-< 

"S 

CD 

CD 

CD 

L 

-4-» 

CD 

O 


M-l 

O 

o' 

o 

o 

CN 


CD 

CD 

CD 

d 

<D 

-M 

CCS 

CD 

?H 

u 


S  ' 


JiO 


C\J 


o 


Created  larger  subset  (>50,000)  of  non-text-based 
trademarks 


Image  Indexing  and  Retrieval 


9 


CM 

CM 


(D 

c/D 

cd 

4-» 

Oj 


V 

V 


O 

to 


d 

<D 

4-> 

X 

O) 


o 

d 

o 

•  rH 

-4-» 

O) 

u 

'Til 

OJ 


cn 

O 

to 

cn 


O)  ^ 


(/) 

(D 

bJD 


'o 

u 

V 

IM 

.s 

d 

o 

d 

V 


o3 

CD 
rd 
bJD 

♦  rH 


(X) 


i 

CJ 

(D 


a> 

•  rH 

-M 

a> 

v-( 

o 

r—H 

O 

u 

"d 

CJ 

> 

o 

?-i 

Ph 


CD 

Ph 

o 

I  i 

CD 

q; 

Q 


U 

d 

c/5 

cn 

<D 

d 

•  fH 

Nl 

a 

bo 

a 

g 


o 

c/5 

CD 

60 

CCS 


•  tH 

O 

f  ( 

o 

u 

60 

d 

c/5 

d 

d 

(D 

CCS 

d 


c/5 


C/5 

O) 

60 

CCS 


d 

O) 

4-* 

CCS 

Ph 

■4-> 

d 

u. 


o 

d 

o 


u 

d 

2  'T^ 

Ph  ^ 


CCS 

d 

CD 


CCS 

CD 


d 

CCS 


?-( 

CCS 

4-> 

CO 


03 

CD 


O) 

?H 

Id 

O) 

CO 

d 

<D 


CO 


O 

d 


CO 

?H 

O) 

> 

4-» 

CO 


CD 

O) 

Q 


CD 

U 

d 

03 

> 

CD 

CO 

03 

CD 


O 

d 


CO 

?H 

CD 

> 

4-> 

CO 

u 


'-M  M-< 

CD  CD 

Ph  p4 

O  O 


CD 

> 

CD 

Q 


cc 

o 


feedback 

Developed  multimodal  demonstration  systems 


•  *  - 

?  Distributed  Retrieval  Architecture 


I 


C/!) 

CD 

on 


C  03 

03 


^  I 

•rH  ^ 

tl  rTi 


u  o;:! 


Hh  cn 
0-) 


2  a;- 

cc 


P  ^ 

.2"^ 

^  ?H 


<v  rC 


O) 

> 


03  .y 

o  ^ 

O  "S 


U  0) 


>  CO 

^  O  g 

-4-»  ?-l  5 


S  « 

s  ^ 


rj  ^ 

”•  o 


^  ^  -rl 
u  60 

-M 

CO  t:5  a; 

2  0)  I 


t3  >^ 

S  'S  ^ 

o3  03  CO 

_ ,  J-i  (U 

<i>  .55 


ft  I  nrt 
X  3  03 


2  o 


S  g 

'-'  o 


Q  -4=:  y 


a 

rS  ^ 
>  o 


a  s 

§D  'S 

CD  Cm 


rx  O) 
pL^  QJ 


^  I 


CC3  43 
CJ  U 


u 

CJ 

cu  y 

I  «  '  ’ 


C4  -+2  S 

p  p 

CJ  CO 


r  )  T-*-i  W 

^  CJ  CO 

P  O) 
H  03  M 


^  O) 

^  f4 
u  _2 

^  CD 
rQ  CD 

rS  ^ 
CD  (/I 


CIIR  1/5/98 


Distributed  Retrieval  Architecture 


I 


U 

(xi 

O 

u 

O. 

Ph 

03 


P  .p  b 

rr^  to  rC 

(D  c/^ 


>  P 

O) 

3  ' 


“  <8 

4-> 

C/!)  ^ 

u 

^  ll  2 

i  »  ^ 

^  -4-*  P-< 

o)  9  ^ 

^  C  ?H 

9r^  ^  Si 

s  «  ^ 

a  o 


C/D 

o; 

c/i 

cd 


<D  ^ 

6J0  CIh 

;-4  u 

o3  O 

— I  u 

d  <1^ 

.5  ^ 

j-i 

CL)  cd 


d  a 

S  ^ 

O  pq 
S  H 

pH  ^ 

^  S 

P  0) 

•  rH  5—1 

X  d 

o)  .5 


86/S/t.  yilO 


Distributed  Retrieval  Architecture 


I 


QJ 


U  ^ 

O  ^ 
^  u 

a;  a 


ig 

SS 

>  s 

Vh  •'^ 


<D 

V 

i2  d 


<1)  n3  On 


C5  JJ 

•SJ  g 


r3  ^ 
43  <u 
u  H 


d)  cd 


o)  q3 
r^  a 


O) 


CD  g 
(D  O 


.2 

d) 


y 


8 

CD 


>  TS 

o  ty 


Q 


g 


CCS  cj 


g  5i 


O)  (_) 

(/) 


CO 

.  CO 


CIIR  1/5/98 


The  INQUERY  Distributed  Architecture: 

Local  Area  Network 


Distributed  Heterogeneous  Systems 


CIIR  1/5/98 


The  INQUERY  Distributed  Architecture: 

Wide  Area  Networks 


GUI 


Presentation  Overview 


03 

C\J 


0) 

•  I— < 

u 


{=; 

O) 

cn 

<D 

u 


o 

a 

O 


Q^  "d 


cn 

O) 

60 

ct3 


03 

O) 


cu 

Q) 

6JD 

cd 


a; 

•4-> 

o3 


o 


<u 

4-» 

c6 

CX 

+-► 

a 

I  ( 

Oh 

60 

0 
•  rH 

cn 

cn 

a> 

u 

o 

Oh 

M-h 

O 


-M  cn 

(t3  0) 


0 

<D 

CD 


s' 

cci 

X 


Oh  CD 

I  I 


oc 

o 


demonstration  of  multimodal  trademark  retrieval 
demonstration  of  feedback  and  retrieval  on  other  images 


s 


^  K 

a»  Oi 

22  ^ 

P  V 


05  m 

5G 


UMass  CIIR,  1997 


Features 


©  UMass  CIIR,  1997 


Features 


©UMass  CIIR,1997 


Automatic  Query  Processing 


a 


<U 

c/5 

ri 

-M 

fci 


o 


13 


a 

O 


a 

cn 

rt 

u 

bJD 

CJ 

•  iH 

cn 


CO 

W) 

O 

T3 


CO 
<U 
CO 
ri 

Pi  ^ 

PL^ 

T5 

TJ 


rtJ 

P 

O 


CO 

W) 

O 

T3 

4-» 

o 

£ 

a 

CO 

Cl. 

=fr 

CO 

W) 

O 

"d 

«4-» 

o 

T 


n3 

a 

O 

c 

CO 


CO 

a 

CO 

CO 

Cd 

'W) 

c 

S 

CO 

(V 

CO 

CO 

'S) 


o 

u 

nj 

(LI 

p 

O) 

WD  a 

W) 

Pi 


% 

c 

« 

% 


cn 

CO 

rt 

C 

S 

CO 


cn 

CO 

'TS 

a 

o 

On  tjo 

e 

o 

u 

< 


T 

CO 

OJ 

CO 

CO 

<« 


CO 


CO 

C 

O) 

X 

CXh 

45 


CO 

'd 

o 

Ph 

g 

o 

u 

'TS 

TJ 


O) 

CO 

(Ti 

CL, 

O) 

CO 

a; 

C 

•  fH 

T— . 

I 

d 

•  iM 

CO 

Vh 

45 

Cl. 

=4: 

CO 

O) 

•w 

CO 

a> 

C 

I 

d 


t 

CO 

O) 

CO 

CO  O) 

45  « 


©  UMass  CIIR,  1997 


©UMass  CIIR,1997 


General  Issues 


C\J 


\ 


U 

< 

u 

O 

•  tH 

u 

o 

MH 

W) 

a 

u 

u 

rs 

O 

(/) 


rt 

o 

nJ 

u 

•  fH 

r— I 

a. 

Cm 

o 

cn 

-4-> 

c 

o 

&H 

ft 


•  vH 

cn 

o 

c 

ft 

> 

O) 

u 

TS 

a 

Ph 


OJ 

;-( 

?S 

«4-» 

ft 

U 

OJ 


CS 

O) 

•M 

ft 

CP 

C 

o 

C 

c 

ft 

> 

a; 

r—H 

O) 

TJ 

e; 

p-i 


o 

ft 

u 

•  fH 

MH 

•  fH 

C/5 

C/5 

ft 

u 


CJ 


u 

a 

u 

u 

O 

u 


C 

O 

•  fH 

ft 

CJ 

•  fH 

"Sh 

Cl, 

ft 

«4-> 

<u 

ft 

CIh 

<u 

•4-* 

S 

o 


c/5 

C/5 

ft 

F^ 

CJ 

rD 

s 

c/5 

T3 

CJ 

ft 

c/5 

CD 

ft 


CJ 

o 

•  1M 

ft 

CJ 

•  fH 

Oh 


O 

•  fH 

ft 

w)  ^ 


•  fH 

C/5 

C/5 

I 


C/5 

C/5 

ft 


a» 


O) 

Vh 

S 

CJ 

CJ 

cn 

c/5 

C/5 

ft 

r-H 

CJ 

fQ 

s 

c/5 


CJ 

o 

•  fH 

ft 
CJ 

•  fH 
MH 

•  fH 

CfJ 
C/5 

ft 

fM 

U 
CD 
U 

U 
<V 

MH 
ft 

C/1 
CD 

u 

^  c 

W)  ?? 


cu 

o 


c/5 

QJ 

C/5 

C/5 

ft 


CJ 

•  fH 

Cf) 

•  fH 

X 

(V 


C 

o 

•  fH 

ft 

N 

•F^ 

c 

ft 

50 

O 

o; 


;h 

cu 

MH 

<u 

u 

CD 

CD 

O 

u 

u 


cu 

s 

50 

•  fH 

CD 

CD 


o 

•  fH 

ft 

N 

•  fH 

d 

ft 

50 

9-1 

o 

a 

cu 

cu 

CJ 

ft 

CD 

cu 

CD 

CD 

ft 


50 

•  fH 

tS 

c 

•  fH 

Ph 


I  I 


©UMass  CIIR,1997 


Three  Classifier  Types 


CO 


CD 

U 


u 

QJ 


•  fH 

CD 

CD 

ri 

u 

O 

bO 

Oi 

iz: 

•M 

U5 

a> 

u 

ri 

O) 

;z; 


cu 


CD 

CD 

fS 

I'—H 

u 

O) 

o 

a 

'C 

a 

0) 

Oh 

O) 

C 

HH 

C 

•  fH 

CD 

<U 


CD 

CD 

nS 

U 


u 

u 

o 

& 

u 

rJ 

TS 

cu 

a> 

pH 

a> 

o 

td 

> 

CJ 

’a 


W  pil 


CD 


<U 


•  tH 
HH 

•  tH 

CD 

CD 

fd 


#  • 


©UMass  CIIR,1997 


K-Nearest-Neighbor  Classifier 


CO 


'5f 


I 


UMass  CIIR,  1997 


Bayesian  and  Relevance  Feedback 

Classifiers 


CO 


LO 


©UMass  CIIR,1997 


Issues  in  K-Nearest  Neighbor 
Classification 


ty>«  ^ 


<u 

cn 

c; 

cr 

CO 

CO 

o 

u 

es 

9h 

•  tH 

O 

-4-» 

MH 

d 

cu 

B 

CO 

<U 

u 

d 

o 

u 

u 

O 

CO 

nd 

to 

n 

d 

}-4 

d 

C 

•M 

rj 

O 

bJD 

CJ 

o 

•  fH 

ffi 

> 

•  vH 

IH 

1 

(d 

g 

T3  g 

0K  \  ^ 


^  CJ 

-  J 

?U|  ^ 


QJ  ^ 

S  Cl!! 

o 

cf)  O 

O)  ^4^ 


j2  ^ 


T3  ^ 

c 

fS  O) 


ui  rt 


^  :s 


O  >t;;J 
cn  S 


fli  ^ 


©UMassCIIR,1997 


Query  Formulation 


CO 

E 

E 

I3' 

Cf) 

■a 

c 

13 

o 

O) 

o 

CO 

CD 


o 

E 

■+-« 

(fi 

jQ 


c 

CO 

■c 

o 

CL 

£ 

00 

o 


cn 

_0 

< 

£ 

•4— > 

I _ 1 

1 _ 1 

<D 

•M 

f- 

- - 

• - ' 

E 

E 

W) 

13 

13 

0) 

CO 

CO 

T— 

Z3 

II 

(/) 

■4^ 

o 

E 

% 

Q> 

CO 

1 

L-i 

QJ 

Z3 

C/5 

-Q 

CO 

a 

©UMass  CIIR,1997 


Query  Formulation  Example 


UMass  CIIR,  1997 


Document  Scores  to  Class  Scores 


UMass  CIIR,  1997 


KNN  Classification  Example: 
Ranked  List  of  Retrieved  Documents 


I 


D 

0 

=tt: 

c/o 

C/) 

a 

1— H 

CO 

u 

T— 

10 

E 

•  • 

C/D 

u 

0 

D 

0 

0 

0 

41= 

D 

0 

0 

0 

> 

0 

+-» 

d 

0 

-4-» 

0 

0 

'4-> 

0 

a 

•E  w 


I  0 

--  0 
j- 

O  0 


0 

^  0 


0  0 


0  ^ 

O  .b 


n  ^ 


•0 

0 

c 

■ji:; 

E 

0 

0 

X) 

’c 

0 

■D 

X 

0 

0 

0 

0 

0 

•  1—1 

0 

p 

H 

1q 

O) 

-3Z  0  0 
0  C  ^ 


o  c 


CD  C 


.E  3E 


^  2 

CO  CD 


00  o 

0  E  g 

o  0  — 

■>  0  c 
0  >^  •— 
X5  0  0 

0)0)0 
C  C  = 

a;  ^ 

2  2  3 

QQ  CD  S 


CO  CM  (M 


000 

CO  CO  00 
CM  CM  CM 


O  O 
00  00 
CM  CM 


^  0 


0 

■o  O 


0  O 


0  0 
0 

C  w 


0  0 
E  .E 
o  > 
0  0 
c  "0 
Q-  o) 

0)  .g 

c  ^ 
=  0 


0  XI 
>  0 
0 


^  S  cn 


0  "O 

CD  < 


CM 

CM  CM 


CD 
CM  CM 


O  O 
00  00 
CM  CM 


O  O 
00  00 
CM  CM 


T-  CM  00 
1-  to  CD 
O  LO  'M- 
CD  h-  LO 
00  CO  O 

to 

LO  m  ID 


CM  C7) 
-r-  O 
O  CO 
CD  CD 
00  ^ 
D 
ID  D 


CO  CD 
T-  CD 
CD  ^ 
D 
CM  O 
D  10 
D  D 


T“  CD 
O 

CO  T- 
CM 

00  00 
10  ID 


5544026  362/1 03  Running  lights  for  in-line  roller  skates 


KNN  Classification  Example: 
Ranked  List  of  Class/Subclass  Candidates 


^UJ  J,0 


JD 

CD 

CO 

C/3 

0 

dD 

CO 

cn 

’u 

cO 

E 

13 

c/3 

CO 


E 

13 

CD 

% 


0 


C/3 

0 

C/3 

C/3 

CO 

0 

MH 

O 

•M 

C/3 


CO 


a  ^ 


_CD 

0 

0 


0 

5 

0 

X3 

o 

0 

0 

E 

*0 

X3 

0 

c 

-1— > 

0 

x: 

•O 

r" 

0 

X3 

0 

V 

-t— < 

5 

0 

H 

X 

LU 

•  ■ 

0 

0 

0 

E 

0 

0 

0 

ZJ 

0 

0 

0 

0 

=tt 

CO 

CO 

CO 

T3 

•o 

T3 

0 

0 

0 

Nfa/ 

-i*; 

0 

0 

0 

0 

0 

0 

0 

0 

0 
-♦— » 

0 

CO 

2  •- 

0  C 

^  0 
O  -J 


0 

0 


■■c 
0 
> 

2  ^ 

O  ^ 

o  o 

'sz  Ic 

0  0 

>  > 

■D  -a 
c  c 
0  0 


0 

o 

u 

CO 


0 

0 

0 

CD 


CM 


h- 


t: 

o 

CL 

CL 

0 

0 

>> 

■D 

O 


x:  0  jc  x: 


vw 

Q. 

Q. 

0 

CD 

C 


.  .  0 

0  0 
0  ^ 

0  XI 

CO  > 

o  c 

Z  .9 

E  I 

0  _9 


00  to  0"  0" 

CM  m  to  to  to 

CO  o  o  o  o 


CM  CD 

^  CO 
1-  T-  O 


O  O  O  O  CM 

00  00  CO  00  CD 

CM  CM  CM  CM  CO 


©UMass  CIIR,1997 


C/D 

4-> 

CD 

GO 

O 

cd 

CZJ 

GO 

C3 


O 

4-> 
•  rH 

GO 

(D 

> 


©  UMASS/CIIR 


Image  Retrieval 


on 

0 

O 


c/5 

<3J 

‘B 

O 

u 


(y:i 

(D 


c/3 

a 

c5 

O 


<D 

c/5 


US 

o 


o 


o 

0 

O 

00 


O. 


0  M 


GO 


s 

(D 
T3 
cd 

ti 

(D 

a 

"H4  ^ 

(D 


o* 

00 


•  ^ 

O 

O 

r— H 

s 

•s 

CO 

C 

bT) 

*  rH 

c/:) 

(D 

<D 

Vh 

<D 


O' 

CO 

Vh 

O 

1— H 

O 

o 

c3 


•  rH 

C/5 


00 

O 

(D 

Vh 

<D 


a  a 
<  < 


o 


T3 

c3 


I  I  I 


a 

> 

D 


<D 

;-i 

o 

.T— ( 
>*— > 

§ 


<D 

00 


c/5 

<D 

4-> 

0 

Xi 

•  tH 

I 

(D 

bO 

c5 


4h 

O 


CD 
& 
00 

■JH 

O 

3  "o 
«  o 

» rH  rs 

^  <D 

§  5 

O  xj 

■7-C  CD 


CD 

O 

§ 

CD 

Ph 

& 


(D 

Pi 


©  UMASS/CIIR 


Example  Pictures 


Overview 


> 

<D 


(D 

U 

<D 

bX) 

cd 


C  bX) 

a 

(D  O 


00 

ci 


cd 

a 


o 

bJO 

ce 


o 

a 

c3 

Cj 

Ph 


bO 

a 

•  ^ 

X3 

o 

c3 


<D 

W) 

a 

a 

•  rH 

(D 

r— H 

O 


i  i 


o 

cd 

(D 

^-1 

(D 

O 

P 

P 

> 

Ij 

Pi 


C/D 
+-» 
r— ( 

C/3 

<D 

Vh 

> 

O 

Vh 

a. 


(D 

T3 
•  ^ 

> 

o 

Cl. 

4-> 

a. 

c 

«  rH 

<D 

00 


> 

0) 


<D 

Vh 

(D 

bX) 


T3 

(D 

GO 

P 

X) 


I  ^ 

'  U 


(D 

C-l 

> 

(D 


(D 

Pi 

4-J 

x 

(D 

H 

'p 

p 

p 

CD 

bX) 

P 


bX) 

•S  ^ 

s  ^ 

i=i  <D 


u  2 


UMASS/CIIR 


People  Involved 


©  UMASS/CIIR 


I 


r  1 

Ph 


> 

)— ( 

.  u 

> 

■  H  ^ 

o  o 

<D 

bJ)  i> 

Cb  ON 

a  w 

•  rH  HH 

■s 

GO 

X)  <u 

8 

0\ 

a  "3  > 

8  SB 

lo  » 
Q-i  I 
<  ' 


o  Cd 
^  ■> 
T3  CD 


-g  ci 

cu 


CD  Cl 
CU  « 
d  •’“' 


CD  CU 
00 


3  ^ 

O 


d 

^  o 

Vh  O 


UMASS/CIIR 


Part  Image  Retrieval 


(D 

cr 

C/D 

(D 

a 


O 

D 

GO 


GO 

<D 

W) 

cd 


CO 

cd 

+-> 


<D 


CO 

•  rH 

a 


c3 

(D 


c/a 

T3 
C 

O 

u 

bJO 

o 

I* 

-b 

•  rH 

■f  a 

^->  c/a 
c/a 

C  0) 

?n  ^ 

bp  (D 

Cd 

'S  ^ 

•S 

•Td  o 

<D 

S 

<D 
(D 

4:5 


CO 

C 

O 


>>  cd 
cd  -rl 

a 

> 

(D 


<D 

b£) 

cd 

s 


^3 

(D 

•  rH 

cr 

(D 

bX) 

CJ 

•  rH 

I 

<D 


O 

>  iz; 


I  I  I 


o 

r-H 

00 


c3 


§ 

cj 

CO 
•  rH 

p 


o 


o 

M-h 

00 

CD 

Cd 


c/a 

'I— > 

c/a 

CD 

a 

-b 

o 

vn 


CO 

bJ)  9h 


Td 

CD 

Td 

CD 

CD 

Oh 

00 


UMASS/CIIR 


Part  Image  Matching 


S.  Chandu  Ravela  Imaga  Retrieval  June  1997  Pg  4. 


Car  Query 


UMASS/CIIR 


Results  of  Car  Query 


Whole  Image  Retrieval 


cn 

<D 

bS) 

cS 


(D 

bS) 

a 


o 

— ( 


P 

a 

"d  g 

S  § 

^  x 

’T;^  O 


GO 

cd 


•r-  ^ 

Ph  ’S 


GO 

(U 

bJ) 

d 


O 

(/} 

'j-i 

c3 

Pi 

<U 

T3 

P 

cj 


(73 

d 

;-H 

3 

d 
4h 

’I-* 

P 

CD 

d 

P  ^ 


(D 


O 

:z; 

•  • 

GO 

CD 


GO 

(D 

bJO 

P 

'i-i 

§ 

> 


P 
O 

bn  OJ 

P  c/3 


P 

P 

> 

'd 

p 

CO 

•  ^ 

p 


CO  1) 

p  ^ 
^  cd 

.3h 

O  P-i 


CO 


5=3 

a> 


P 

^  S 

X  o 

p 


CD 

O 

O 

bX3 

(D 

P 


<D 

Vh 

3 

cd 

& 

5 


I  i  I 


©  UMASS/CIIR 


Moments 


©  UMASS/CIIR 


Jpeg  Matching. 


i 


\ 


00 

00 

o 

4-> 

a 

Td 

(D 

•  rH 
>' 

« 

Td 


v-i 

<2 

(D 

r~H 

r— H 

•  rH 

a 

00 
+- > 

sd 

(D 

•  rH 


bO 

a 

i 

I" 

o  5=! 
^  cE 

^  on 

■°  G 


<D 

bO 

o 

bO 

o 

r^ 

•  rH 

;  rH 

5+-' 

o 

cd 

Cl 

•  tH 

4h 

'Td 

P 

C3 

g 

• 

<D 

O 

n 

•  rH 

CD 

o 

o 

Qh 

•  f-H 

b£) 

c/3 

O 

o 

bX) 

3 

rP 

o 

O 

bX) 

00 

(D 

H 

<D 

o 

<D 

o 

a 

CD 

u 

3 

Ph 

cd 

<D 

s 

•  1 — ) 

O 

o 

c/3 

cd 

CD 

c/3 

T3 

CJ 

rA  ^ 

I  o 

o  -§ 

X)  ^ 

;-i 

(D 
> 
O 


di 

Uf 


^3 

</) 


C3 

<U 


a 


o 


id 


<D 


UMASS/CIIR 


©  UMASS/CIIR 


Relevance  Feedback 


(D  5^-1 

Q  8 


© 


UMASS/CIIR 


Se&rch 


Local  Curvature 


UMASS/CIIR 


Curvature  Matching 


©  UMASS/CIIR 


Curvature  -  Results. 


©  UMASS/CIIR 


Curvature  -  Results 


Curvature  -  Results 


©UMASS/CIIR 


Color  Image  Retrieval 


•  O 

52  0-(  oi 

5  ^  ^2 

d  tu) 


<1^ 

O  cd 


O  ^  o 

o  J!  -S 


•  5^ 

w  o 

d  >» 

•y  d 


8 

<D  ^ 
>  W) 
'd  d 

'd  ^ 


^  "S  ^ 

o  ^ 

o  ?? 

^  c/3  d 

<D  d  !d 

■^i  1d  o 

d  d  <D 

Q^  Ph 

S  S 

o  o  I 

u  u  ' 


b 

<D 

S'  (0 

^  J 

O  d 

dl 

P.  ^ 

00  Q 


B  ^ 

o  ^ 

O  d 
d"  CO 


cd 

<D 

d 

.  d 

03  bO 


J^t0« 


\m. 


OLOOIOOIOOLOO 
^  CO  CO  CM  CM  T-  1- 


UMASS/CllR 


Results  of  Color  Retrieval 


©  UMASS/CllR 


Plant  Patents 


ff  -2 


ttoii  ^ 


o 

2  ^ 
Qh  ^ 
Oh  O 

<!  q=l 

<D  ^ 

^  o 

r2  jh 

GO  o 

o  "o 

Oh  U 


S 

o  O 


S  c/2 

O  tD 
C  O 

tH  4— > 

H  <D 


o  ^  S 

-2  cd  a 


u  4h 

_2  5^ 


o  o  ^ 

o  u 


•  fn  T3 
o  bX) 


^  cd 

cd  O  O 

1—1  1— <  5— ( 

(:l^  p-^  fin 


©  UMASS/CIIR 


Plant  Patents 


©  UMASS/d 


UMASS/CilR 


Image  and  Text  Retrieval 


a 


©  UMASS/CIIR 


Image  and  Text  Retrieval 


c/3 

a 


cd 

O 

4h 

•  i~( 

00 

c/3 

cd 

Tj 

1 


<D 
"d 

W) 


c/3 

d 

cd 

> 

<D 


<D 

Td 

•  rH 


CD 

3 

cd 

> 

d 

o 

bn 

d 

•  rH 
CO 

d 

. . i 

cd 

'B 

CD 

bn 

cd 

a 


<2 


bn 

cd 


(D 

:S 

CD 

-s 

a 

o 

CD 

00 


<D 

CO 

cd 

Ph 

'd 

P 

cd 


Ph 

CD 

-i-> 

sd 

o 

d 

o 

■H» 

^3 

a 

o 

o 

00 

<D 

U 

O 

o 

c/3 


ID 

d 

D 

X 

D 

d 

p 

•  rH 


I 


©  UMASS/CIIR 


Future  Work 


a 


o 

c/3 


c/3 

CD 


c3 

H 


c/3 


x 

CD 


c/3 

CD 

u 

O 

O 

c/3 


a 

> 

CD 


CD 

;h 


bO 

CD 


X 

(D 


J=3 

cd 


^  2?n 

5Zj  bU 


cd 


bX) 

a 


^  ’S 

I  "s 

S  6 


o 

cd 

•  ^  ^ 

(D  CD 
■  th  CD 


'TD  CD 
O  c/3 


CD 

GO 

-§ 


<o 

B 


a 


T3 

CD 

v-t 

O 


CD 


bX) 
CD  a 


c/3 


o 


§ 


Oh  T3 

rH  ^ 

O 

3  o 

S 


c/3 

CD 

bX) 

c5 

a 


GO 

<D 

bX) 

ccj 

s 


o 

CD 

a 

c 

o 

o 


CD 

4h 

O 


ccJ 


CD 

O 

fl 

a 

<D 

Oh 


> 

O 


a 

cD 
M  ^ 

A  \ 


o 


H-c 


tD 

CD  i-rj 


CD 

(D 


;-i 

c2 


C^  o 


CD  _ 
O  CD 


a 

GO 

CD 

c3 

&: 

o 

GO 

b£) 

> 

CD 

C4-H 

• 

c3 

1 

•  rH 

')— > 
cj 

y  H 

CD 

o 

<D 

4h 

?" 

o 

s 

o 

o 

CD 

sb 

V-i 

CD 

O 

CD 

CD 

00 

1 

1 

o 

c/3 

• 

• 

# 

cj 

<D 


b£) 


CD 

< 


3 

@ 


Improve  speed  of  part  image  techniques. 


